J 



Europaisches Patentamt 
European Patent Office 
Office europ^en des brevets 



© Publication number 



0 456 103 A2 



EUROPEAN PATENT APPLICATION 



© Application number: 91107061.3 
® Date of filing: 02.05.91 



© int.ci.5-.A61B 19/00 



© Priority: 11.05.90 US 523611 


880 Alvarado Ave., Apt. 216 


© Date of publication of application: 


Davis, California 95616(US) 


Inventor: Mittelstadt, Brent D. 


13.11.91 Bulletin 91/46 


2761 Tunnel Street 


© Designated Contracting States: 


Placerville, California 95667(US) 


Inventor: Musits, Bela Louis 


DEFRGB 


42 East Hook Cross Road 


© Applicant: International Business Machines 


Hopewell Junction, N.Y. 12533(US) 


Inventor: Paul, Howard A. 


Corporation 


7170 Morningside Drive 


Old Orchard Road 


Loomis, California 95650(US) 


Armonk, N.Y. 10504(US) 


Inventor: Taylor, Russell Highsmith 




21 Adams Road 


@ Inventor: Glassman, Edward 


Ossining, New York 1 0562(US) 


207 E. 74th Street 




New York, New York(US) 


© Representative: Blutke, Klaus, Dipl.-lng. 


Inventor: Hanson, William Arthur 


3390 Await Drive 


IBM Deutschland GmbH Intellectual Property 


Mountain View, Calif. 94040(US) 


Dept. Schtinaicher Strasse 220 


Inventor: Kazanzides, Peter 


W-7030 Boblingen(DE) 



< 

CO 

o 

CO 

If) 



© Image-directed robotic system for precise surgery. 

© A robotic surgical system (10) includes a mul- 
tiple degree of freedom manipulator arm (14) having 
a surgical tool (22). The arm is coupled to a control- 
ler (24) for controllably positioning the surgical tool 
within a three dimensional coordinate system. The 
system further includes a safety monitoring proces- 
sor (38) for determining the position of the surgical 
tool in the three dimensional coordinate system rela- 
tive to a volumetric model. The volumetric model 
may be represented as a constructive solid geome- 
try (CSG) tree data structure. The system further 
includes an optical tracking camera system (28,32) 
disposed for imaging a region of space that includes 



at least a portion of the manipulator arm. An output 
of the camera system is coupled to the processor 
(38) that processes the volumetric model for deter- 
mining if the surgical tool is positioned outside of the 
volumetric model. The system further includes a 
strain gage (40) for detecting slippage in three di- 
mensions between an immobilized tissue, such as 
bone, and a reference point (44). The system also 
includes multiple and redundant safety features for 
suspending a motion of the surgical tool to prevent 
the tool from operating outside of the predetermined 
volume of space. 
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This invention relates to a robotic system that 
integrates an interactive Computed Tomagraphy 
(CT)-based presurgical planning component with a 
surgical system that includes a multiple-degree of 
freedom robot and redundant motion monitoring. 
An illustrative application is presented in the con- 
text of a system that prepares a femoral cavity to 
have a shape precisely determined for receiving a 
cementless prosthetic hip implant. 

It has been found that computed tomagraphy 
(CT) imaging and computer modelling methods 
provide a precision for pre-surgical planning, sim- 
ulation, and custom implant design that greatly 
exceeds the precision of subsequent surgical ex- 
ecution. For example, approximately one half of the 
300,000 total hip replacement operations per- 
formed each year use cementless implants. Stabil- 
ity of the implant, uniform stress transfer from the 
implant to the bone, and restoration of the proper 
biomechanics critically affect efficacy and, in turn, 
are significantly affected by the proper placement 
of the implant relative to the bone. An important 
factor in achieving proper placement of the implant 
is the accuracy with which the femoral cavity is 
prepared to match the implant shape. 

Recently reported research confirms that gaps 
between implant and bone significantly affect bone 
ingrowth. One study of the standard manual 
broaching method for preparing the femoral cavity 
found that the gaps between the implant and the 
bone is commonly in the range of one millimeter to 
four millimeters and that the overall resulting hole 
size was 36% larger than the broach used to form 
the hole. As a result, only 18-20 percent of the 
implant actually touches bone when it is inserted 
into such a hole. Furthermore, the placement of the 
implant cavity in the bone, which affects restoration 
of biomechanics, is as much a function where the 
broach "seats" itself as of any active placement 
decision on the part of the surgeon. 

Typically, precise surgical execution has been 
limited to procedures, such as brain biopsies, for 
which a suitable stereotactic frame is available. 
However, the inconvenience and restricted applica- 
bility of these devices has led some researchers to 
explore the use of robots to augment a surgeon's 
ability to perform geometrically precise tasks 
planned from CT or other image data. 

Safety is an obvious consideration whenever a 
moving device such as a robot is used in the 
vicinity of a patient. In some applications, the robot 
does not need to move during the "in-contact" part 
of the procedure. In these applications the robot 
moves a passive tool guide or holder to a desired 
position and orientation relative to the patient. 
Brakes are then set and motor power is turned off 
while a surgeon provides whatever motive force is 
needed for the surgical instruments. Other surgical 



applications rely on Instrumented passive devices 
to provide feedback to the. surgeon on where the 
instrument is located relative to an image-based 
surgical plan. 

5 In an IBM Research Report (RC 14504 

(#64956) 3/28/89) R.H.Taylor et al. describe a 
robotic system for milling a correctly shaped hole 
into a femur for receiving a cementless hip implant. 
The system computes a transformation between 

to CT-based bone coordinate data and robot cutter 
coordinates. The transformation is accomplished in 
part by a combination of guiding and tactile search 
used to locate a top center of each of three align- 
ment pins that are pre-surgically affixed to the 

75 femur and CT-imaged. This robotic system in- 
cludes a vision subsystem to provide a redundant 
check of the robot's motion to ensure that the tool 
path does not stray outside of a planned work 
volume. An online display is provided for the sur- 

20 geon to monitor the progress of the operation. 
Proximity sensors may be positioned to detect any 
subsequent motion of the pins relative to a robot 
base. 

It is the object of the invention to provide a 

25 robotic surgical system particularly in conjunction 
with an apparatus for preparing a carity within a 
bone to receive a prosthetic implant, and a method 
of surgically removing a quantity of tissue from a 
patient, as well as a surgical planning system. 

30 The invention discloses a robotic surgical sys- 
tem that includes a multiple degree of freedom 
manipulator arm having a surgical tool. The arm is 
coupled to £ controller for controllably positioning 
the surgical tool within a three dimensional coordi- 

35 nate system. The system further includes appara- 
tus for determining the position of the surgical tool 
in the three dimensional coordinate system relative 
to a volumetric model. The determining apparatus 
includes a device for detecting a location of the 

40 surgical tool, such as an optical tracking system 
disposed for imaging a region of space that in- 
cludes at least a part of the manipulator arm. An 
output of the tracking system is coupled to a pro- 
cessor that processes the volumetric model for 

45 determining if the surgical tool is positioned outside 
of a predetermined volume of space. The system 
further includes redundant safety checks including 
a strain gage for detecting in three dimensions any 
slippage between an immobilized tissue, such as 

so bone, and a reference point and also a force sen- 
sor coupled to the surgical tool. The multiple and 
redundant safety devices are employed for sus- 
pending a motion of the surgical tool to prevent the 
tool from operating outside of the volumetric 

55 model. Trie coordinates and structure of the volu- 
metric model are determined during a pre-surgical 
planning session wherein a surgeon interactively 
selects and positions a suitably shaped implant 
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relative to images of the bone within which the 
implant is to be installed. 

The above set forth and other features of the 
invention are made more apparent in the ensuing 
Detailed Description of the Invention when read in 
conjunction with the attached Drawing, wherein: 
Fig. 1 is a block diagram showing a presently 
preferred embodiment of a surgical robotic sys- 
tem; 

Rg. 2a illustrates a human hip prosthetic im- 
plant; 

Rg. 2b illustrates the implant of Rg. 2a im- 
planted within a femur, the Figure also showing 
the placement of three alignment pins upon the 
femur; 

Rg. 3a illustrates a method of determining a 
cutter work volume for the implant of Fig. 2a; 
Fig. 3b is a CSG tree representation of the 
cutter work volume of Fig. 3a; 
Fig. 4 is a graph of measured force, resolved at 
the cutter tip, as a function of time, the graph 
further illustrating first and second force thresh- 
olds; 

Fig. 5 illustrates information flow during pre- 
surgical planning; and 

Rg. 6 is a block diagram showing in greater 
detail the pre-surgery system and the coupling 
of the pre-surgery system to the robot controller 
and on-line display. 
Rg. 1 is a block diagram illustrating a present 
embodiment of a surgical robotic system 10. Sys- 
tem 10 includes a robot 12 having a manipulator 
arm 14 and a base 16. The arm 14 has at least 5- 
axes of motion and includes a plurality of joint (J) 
motors J1, J2. J3 and J4 that provide four degrees 
of motion and, in the present embodiment, a pitch 
motor (PM) 18 that provides the fifth degree of 
motion. In the present embodiment of the invention 
the robot 12 is a four-degree of freedom IBM 7575 
or 7576 SCARA manipulator having an additional 
pitch axis (IBM is a registered trademark of the 
International Business Machines Corporation). The 
PM 18 is presently implemented with a stepper 
motor having an encoder, primarily for stall detec- 
tion, that is controlled with a commercially available 
indexer of the type that accepts a user provided 
acceleration, velocity and distance commands. 
Each of the manipulator arm joints has an asso- 
ciated controlling microprocessor. 

The robot 12 further includes a six degree-of- 
freedom wrist-mounted force sensor 20. One suit- 
able force sensor is known as a Lord Force/Torque 
Wrist Sensor, Model F/T-30/100 having a maximum 
force limit of 30 pounds and a resolution of 1/40 
pound. The force sensor 20 is coupled via a serial 
or a parallel interface to a robot controller 24. 

The robot 12 further includes an end effector 
having a cylindrical high-speed (65000 rpm) pneu- 



matic surgical cutting tool 22. During surgery all 
but the robot's end-effector are covered by a sterile 
sleeve, the end-effector being separately sterilized. 
The robot 12 is positioned relative to an operating 
5 table such that it has ready access to the surgical 
region. 

The robot controller 24 provides servocontrol. 
low-level monitoring, sensor interfaces, and higher- 
level application functions implemented in the 

10 AML/2 language. The controller 24 is presently 
embodied in an industrial IBM Personal Computer 
AT data processor wherein low-level servo control 
and force sensor interface is provided by suitable 
printed circuit cards that are plugged into the pro- 

75 cessor bus (Personal Computer AT Is a registered 
trademark of the International Business Machines 
Corporation). Controller 24 includes an AM172 Lan- 
guage Interpreter and also Motion Control System 
(MCS) software. A commercial version of this soft- 

20 ware is described in "AML-2 Language Reference 
Manual", Manual #G7X1369 and in "AML-2 Manu- 
facturing Control System User's Guide", Manual 
#G67X1370. both of which are available from IBM 
Manufacturing Systems Products, Boca Raton, 

25 Rorida. 

In the present embodiment the AML/2 software 
is modified to accommodate the operation of the 
PM 18 and the force sensor 20. During surgery, the 
force sensor 20 is employed in conjunction with a 

30 Force Monitoring Processor (FMP) 53 to support 
redundant safety checking, tactile searching to lo- 
cate aligning pins, and compliant motion guiding by 
the surgeon. 

The FMP 53 is interfaced to the wrist-mounted 

35 force sensor 20 and computes forces and torques 
resolved at the cutter 22 tip. As can be seen in the 
graph of Fig. 4 if any cutter 22 tip force component 
greater than approximately 1.5 kgf (Lt) is detected, 
the robot controller 24 is signalled to freeze motion. 

40 Forces greater than approximately 3 kgf (L2) result 
in arm power being removed, an arm 14 
"shutdown" condition. The force sensor 20 is effec- 
tive in detecting such conditions as the cutter 22 
stalling, encountering improperly excised soft tis- 

45 sue. and changes in bone hardness such as that 
which occurs between trabecular and cortical bone. 

Also coupled to the controller 24 is a hand-held 
pendant 26 for use by the surgeon as an input 
terminal as will be described. 

50 A motion monitoring subsystem includes, in the 
present embodiment, an optical tracking system 
having a camera 28 with three spatially separated 
image sensors 30a. 30b and 30c. Coupled to cam- 
era 28 is a camera processor 32 that visually tracks 

55 in three dimensions the position and orientation of 
a plurality of infrared (IR) beacons, such as LEDS 
34. that are mounted on a reference plate 36 
coupled to the robot end effector. The optical track- 
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ing system is but one of several redundant motion 
detecting systems employed during the cutting 
phase of the surgery to verify that the cutter 22 tip 
does not stray more than a specified amount out- 
side of a defined implant volume. In a presently 
preferred embodiment of the invention the optical 
tracking system is a type known as Optotrak that is 
manufactured by Northern Digital, Inc. The three 
image sensors 30a-30c are line-scan devices 
mounted in a rigid frame which track the position of 
IR LEDs 34 in three dimensional space to an 
accuracy of 0.1 mm/m 3 . 

In the present embodiment the reference plate 
36 includes eight LEDs 34 which function as posi- 
tional beacons. Camera processor 32 software is 
employed to compute a camera-based coordinate 
system from the beacon locations. Robot-to-cam- 
era and cutter-to-reference plate transformations 
are computed by a least squares technique from 
data taken with the robot arm 14 in various known 
positions, using appropriate linearized models. 

An output 32a of the optical tracking system is 
coupled to a safety monitoring processor 38 that 
has as one function a task of verifying that the 
cutter 22 remains within the predetermined spatial 
volume associated with the selected implant. Pro- 
cessor 38 receives a coordinate transformation 
(Tcp) of the reference plate 36 relative to camera 
28 from the camera processor 32. In an alternative 
embodiment, processor 38 receives the coordi- 
nates of LEDS 34 from camera processor 32 and 
computes the coordinates of reference plate 36 
itself. 

During a calibration phase prior to surgery, 
robot controller 34 moves the robot to a plurality of 
positions and orientations. After each motion, robot 
controller 24 transmits the position and orientation 
of the robot's cutter 22 to processor 38 over a 
communication bus 24a. Processor 38 uses this 
information, together with the reference plate 36 
coordinates relative to the camera 28, to compute 
the coordinate transformation (Trc) between the 
camera coordinate system and the robot coordinate 
system and also to compute the coordinate trans- 
formation (Tpk) between the cutter 22 and the 
reference plate 36. As a result, processor 38 is 
enabled to determine the coordinate transformation 
(Trk) of the robot's cutter 22 relative to the robot 
from the relationship 

Trk = Trc* Tcp* Tpk. (1) 



In the presently preferred embodiment, the 
calibration for Tpk is accomplished by placing plate 
36 in many orientations with respect to camera 28 
with the tool tip being maintained in the same 
location. This may be accomplished either by reli- 



ance on the robot-to-tool calibration or preferably 
by means of a tactile search procedure using force 
sensor 20 to locate the cutter tip at a known 
constant position relative to a calibration pin or post 

5 54. Alternative embodiments include the use of 
other sensing means either for direct measurement 
of the cutter tip position or as feedback allowing 
the robot controller 24 to place the cutter tip in a 
known position relative to a reference landmark or 

;o coordinate system. 

In other embodiments of the invention the posi- 
tion of the manipulator arm 12 may be tracked in 
three dimensions by, for example, magnetic sens- 
ing devices or by an ultrasonic ranging system. 

75 That is, the practice of the invention is not limited 
to use with an arm motion detector that relies on 
detecting optical beacons. 

During surgery processor 38 receives inputs 
over communication bus 24a from the robot con- 

20 troller 24 specifying, for this embodiment, a Con- 
structive Solid Geometry (CSG) tree representation 
of the volume to be checked ("check volume") and 
the coordinates of this volume relative to the robot 
12. Processor 38 repeatedly receives reference 

25 plate coordinates (Tcp) from camera processor 32 
or, alternatively, receives the coordinates of LEDS 
34 and computes (Tcp). Processor 38 computes 
Trk and determines if the cutter 22 is within the 
specified check volume. 

30 Processor 38 also monitors a bone slippage 
detector which, in accordance with an aspect of the 
invention, is comprised of strain gages 40 which 
are physically coupled to a tissue, such as a bone 
42, that is being surgically altered.. The strain 

35 gages 40 are disposed to measure in three dimen- 
sions any displacement of the bone 42 relative to a 
bone fixator 44, the fixator 44 being rigidly coupled 
to the robot base 16. The strain gages 40 are 
interfaced through appropriate circuitry, including 

40 an analog-to-digital converter, to the processor 38. 

It has been demonstrated that motions on the 
order of 0.1 mm are readily detectable in this 
manner. Furthermore, it has been determined that 
with a suitable fixator 44, such as a device that 

45 employs screw-type connections made directly to 
the bone, that even rather large forces (5 kgf) 
produce only a few microns of motion. Bone mo- 
tion of this small magnitude is negligible in the 
context of this application. Thus, although no sig- 

50 nificant bone motion is expected during surgery, 
the strain gages 40 provide an an immediate in- 
dication if any bone slippage should occur. 

If slippage of the bone is detected at least two 
options are available. A first option is to recalibrate 

55 the system by relocating the position of the bone in 
space by locating the three pins 46 with the robot 
effector in accordance with a procedure described 
below. A second option is employed if the slip 
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sensor is accurately calibrated. The second option 
involves a mathematical determination of the 
amount of bone slippage to derive a compensation 
factor that is applied to subsequent robot motions. 
The first option is preferred for simplicity. 

Further in accordance with the invention, and 
as illustrated in Figs. 3a and 3b, the safety monitor- 
ing processor 38 employs a volumetric processing 
technique to verify that the cutter 22 tip does not 
stray by more than a predetermined tolerance be- 
yond a spatial envelope that corresponds to the 
three dimensional implant model. The present em- 
bodiment of the invention employs the above men- 
tioned CSG tree "check volumes", corresponding 
to shapes resulting from implant and cutter selec- 
tion, that are constructed from primitives bounded 
by quadric surfaces located at a defined distance, 
such as one millimeter, outside of the furthest 
nominal excursions of the cutter 22. In Fig. 3a the 
volume of space that corresponds to a selected 
implant shape is determined by partitioning the 
implant shape into a plurality of primitive shapes 
that correspond to (a) a cutter approach volume, 
(b) an implant proximal portion and (c) an implant 
distal portion. The inner dashed line corresponds to 
the maximum cutter 22 excursion as measured 
from the center (X) of the cutter 22. The cutter 22 
outer edge is thus coincident with the outer, solid 
envelope of the implant volume. In practice, the 
cutter center (X) is uniformly offset away from a 
longitudinal axis of the implant shape by the pre- 
determined cutter excursion tolerance, such as one 
millimeter. 

In accordance with CSG technique the implant 
primitive shapes are organized into a tree structure 
(Fig. 3b). A CSG tree has internal nodes that repre- 
sent Boolean operations and rigid motions while 
the leaves of the tree represent solid primitives, 
here representative of the implant proximal and 
distal portions and the approach volume. The robot 
controller 24 can specify the check volumes to the 
safety monitoring processor 38 in at least two man- 
ners. A first approach specifies a check volume for 
each major phase of the surgery, such as specify- 
ing the check volume for the approach volume, 
then the proximal portion of the implant and subse- 
quently specifying the check volume for the distal 
portion. A second approach increases the number 
of leaves of the CSG tree by disassociating each 
major portion into its constituent required cutting 
motions and specifying a check volume for each 
cut (CUT1-CUTn). A combination of these two ap- 
proaches can also be used. Furthermore, each 
check volume may be specified immediately prior 
to use or may be specified at any time after the 
robot-to-bone transformation has been computed 
and stored for subsequent selection and use. 

During surgery the position of the LED bea- 



cons are observed and, via the transformations 
discussed above, the location of the center of the 
cutter 22 is determined relative to the implant vol- 
ume by searching and processing the CSG tree 

5 structure by known techniques. If the cutter 22 is 
determined to stray outside of the implant volume 
a command is sent by the safety monitoring pro- 
cessor 38 to the robot controller 24 to freeze mo- 
tion. This command is preferably sent via a mini- 

io mum latency path 24c t such as an optically iso- 
lated digital port. After freezing motion the robot 
controller 24 interrogates the safety control proces- 
sor 38 via the serial communication link 24a for 
more detailed information regarding the "out-of- 

75 bounds" condition. 

It has been determined that this technique reli- 
ably detects a motion that crosses a predetermined 
CSG threshold to within approximately 0.2 mm 
precision with constant orientation, and approxi- 

20 mately 0.4 mm precision with cutter reorientation. 
Cutter motion checking rates of approximately 3-4 
Hz are obtained using the present embodiment. At 
typical bone cutting speeds a total cutter excursion 
before motion is frozen is approximately two mil- 

25 limeters, after all system latencies are accounted 
for. 

The surgeon-system interface further includes 
an online display system that includes a high reso- 
lution monitor 48 coupled to a display processor 

30 50. The display processor 50 receives information 
from the robot controller 24. the safety monitoring 
processor 38 and a CT pre-surgery system 52. 
This information is combined by the display pro- 
cessor 50 to visually depict the progress of the 

35 cutting procedure by superimposing the CT-de- 
rived bone images and a corresponding cross- 
sectional view of the selected implant shape. The 
CT images may include both transverse and lon- 
gitudinal images of the femur having appropriate 

40 cross-sectional views of the selected implant 
superimposed thereon. A particular cross-sectional 
bone image that is selected for display typically 
corresponds to the current depth of the cutter 22 
within the femur. 

45 The pendant 26 is a gas-sterilized hand-held 
data terminal that allows the surgeon to interact 
with the system 10 during the course of the opera- 
tion. Menus invoked from the pendant 26 and dis- 
played on the display 48 permit the surgeon to 

so interrogate system 10 status, to select local actions 
such as manual guiding or withdrawal of the cutting 
tool, to continue a present motion, to discontinue or 
repeat a present step of the surgical procedure, or 
to restart the procedure from some earlier step 

55 altogether. One very common case is a simple 
"pause" function that inhibits further forward mo- 
tion of the cutter 22 to allow the surgeon to satisfy 
himself that all is well or to permit the performance 
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of some housekeeping function such as refilling an 
irrigation bottle. Pendant 26 also supports an emer- 
gency power on/off function and may also be em- 
ployed to control the overall sequence of applica- 
tion steps and to select appropriate preprogram- 
med error recovery procedures should the need 
arise. 

Each of the major system components, that is 
the robot controller 24, FMP 53 and safety monitor- 
ing processor 38, are enabled to freeze (inhibit) all 
robot motion or to turn off manipulator and cutter 
power in response to recognized exception con- 
ditions. If one of these conditions occurs the sur- 
geon must explicitly re-enable motion from the 
hand-held pendant 26. 

During surgery the principal safety monitoring 
requirements are (1) that the robot 12 should not 
"run away"; (2) that it should not exert excessive 
force on the patient; (3) that the cutter 22 should 
stay within a pre-specified positional envelope rela- 
tive to the volume being cut; and (4) that the 
surgeon be able to intervene at any time to stop 
the robot 12. Once robot motion is stopped the 
surgeon is able, via the pendant 26, to query the 
robot's status, to manually guide the effector, to 
select an appropriate recovery procedure to con- 
tinue the surgery, or to completely terminate use of 
the robot 12 and continue manually with the sur- 
gery. 

As a part of the redundant safety monitoring of 
the system 10 the robot controller 24 itself rou- 
tinely performs many safety and consistency 
checks, including such functions as determining 
position and velocity deadbands of the joint servos, 
monitoring of external signals, and the maintenance 
of a safety timeout monitor (STM) 24b which re- 
moves arm 14 power if the controller 24 does not 
affirmatively verify system integrity at predeter- 
mined intervals. In a presently preferred embodi- 
ment the interval is 16 ms. In addition to a robot 
power-enable relay (not shown) the controller 24 
software provides facilities for disabling manipulator 
power, for "freezing" motion, for resuming inter- 
rupted motions, and for transferring control to pre- 
determined recovery procedures. A number of con- 
ditions, including externally signalled consistency 
checks, the exceeding of force thresholds, push- 
button closures and the like interrupt the robot 
controller application program and cause motion to 
be frozen or, occasionally, arm 14 power to be 
removed. 

In accordance with the redundant safety check- 
ing of the system 10 the combination of the arm 
position sensing system, including the camera pro- 
cessor 32, and the safety monitoring processor 38 
provide an independent verification that the cutter 
22 tip remains within a defined volume relative to 
the bone. This is accomplished by (a) verifying with 



the strain gages 40 that the bone does not move 
relative to the fixator 44, which is rigidly affixed to 
the robot base 14, and (b) verifying that the arm 14 
end effector never strays from the defined volume 

5 in space. 

The CT pre-surgery system 52 enables the 
surgeon to select a desired shape for the prosthetic 
implant 1 (Fig. 2a) and by interactively placing the 
implant shape upon the CT-generated images of 

10 the femur to determine where the required cavity is 
to be machined into the femur 2 (Fig. 2b). The pre- 
surgery system 52 also provides a volumetric 
model corresponding at least in part to the shape 
of the implant, the volumetric model being com- 

15 puter maintained and processed in order to ac- 
curately machine the implant-receiving cavity into 
the femoral canal. Rg. 2b illustrates the implant 1 
in relation to the femur 2, the pelvis 4 and the 
acetabular cup 5. In a presently preferred embodi- 

20 ment of the invention the pre-surgery system 52 is 
implemented on an IBM PS/2-80 workstation coup- 
led to an augmented display (PS/2 is a registered 
trademark of the International Business Machines 
Corporation). 

25 Reference is now also made to Figs. 5 and 6 
wherein Fig. 5 illustrates information flow during 
pre-surgical planning and Fig. 6 depicts in block 
diagram form and in greater detail than Rg. 1 the 
coupling of the pre-surgery system 52 to the robot 

30 controller 24 and on-line display processor 50. Pri- 
or to the prosthetic implant surgery the plurality of 
locating pins 46 are inserted into the greater 
trocanter and femoral condyles. A CT scan is made 
of the leg to generate CT data (Block A). The pre- 
ss surgery system 32 processes the CT data to pro- 
duce a three dimensional CT image. The pre- 
surgery system 52 subsequently processes the CT 
image (Block B) to locate the top center of each 
alignment pin 46 relative to the CT coordinates. 

40 thereby determining a set of bone reference co- 
ordinates. The surgeon interactively selects appro- 
priate orthographic cross sectional slices through 
the CT image of the bone. After selecting the 
appropriate implant shape (Block C) from an im- 

45 plant shape library (Block D) the surgeon inter- 
actively positions the implant model relative to the 
CT image as a function of bone density and other 
factors while the pre-surgery system 52 generates 
cross-sectional views and develops three dimen- 

so sional models of the implant and the femur (Blocks 
E and F). The pre-surgical system 52 also supports 
an optional interactive custom design (Block G) of 
the prosthetic implant. This is accomplished by the 
surgeon entering salient structural characteristics of 

55 the femur. The resulting custom implant shape is 
stored within the implant designs library (Block D). 

After the surgeon has placed the implant shape 
at a desired position wfthin the CT image of the 
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femur 2 the coordinates of each pin 46, together 
with the implant identification and the surgeon- 
specified implant location, are stored within a sur- 
gical data file (Block H). Three dimensional implant 
and bone model data generated by the modeler 
(Block E) is provided to the display processor 50 
together with three dimensional image data from 
Block B. The surgical data file information is pro- 
vided to the robot controller 24 so that the robot 
controller 24 is enabled through various "part sub- 
routines" to determine during surgery the se- 
quence of effector motions required to form the 
implant-shaped cavity at the surgeon-specified lo- 
cation within the femur. In an alternative embodi- 
ment the required effector motions are pre-com- 
puted relative to the pin 46 location Information 
contained within the surgical data file. The pre- 
computed cutter motions are stored and may be 
made available, before surgery, to the safety moni- 
toring processor 38. In either case the robot con- 
troller 24 provides the surgical data file information 
to the safety monitoring processor 38 for verifica- 
tion that the cutter 22 Is operating within the spatial 
volume associated with the selected prosthesis 
shape and also for display to the surgeon so that 
the surgeon can monitor the progress of the cutter 
22. 

After the pre-surgical planning procedure de- 
scribed above the surgery proceeds as follows. 

The patient is brought into the operating room 
and the surgery proceeds in a conventional manner 
until the point where the femorai cavity must be 
created. That is, the hip is opened and dislocated, 
the ball of the femur is cut off, and the acetabluar 
cup 5 is installed. 

The three aligning pins 46 are exposed and the 
bone is placed into the fixator 44 to lock the femur 
firmly to the robot base 16. If used, the strain 
gages 40 are physically coupled to the bone. 

A combination of guiding and tactile search is 
then employed to locate the top center of each 
aligning pin 46 in the robot's coordinate system. 
This data is used to compute the transformation 
between CT-based bone coordinates and robot cut- 
ter 22 coordinates. 

To accomplish this pin-locating procedure a 
calibration ball probe is placed in the cutter 22 
collet. The surgeon manually guides the robot 12 
so that the ball probe tip is located approximately 
at the top center of one of the pins 46. This guiding 
may be done either from the pendant 26 or by 
force compliance. In force compliant guiding, the 
surgeon grasps any part of the robot that is distal 
to force sensor 20. The robot controller 24 moves 
the robot in response to forces detected by the 
force sensor 20. In the presently preferred embodi- 
ment forces and torques are resolved to a coordi- 
nate system aligned with cutter 22 and the velocity 



of motion is proportional to force. Alternative em- 
bodiments may employ other force compliance 
laws. The surgeon then manually guides the robot 
so that the probe tip is located a few millimeters 

5 back along the pin 46 axis. From this information, 
the robot controller 24 computes the estimated 
center and normal of the pin 46. Robot controller 
24 next locates three points on the top of the pin 
46 by a tactile search with the force sensor 20 and 

;o computes the exact pin normal. Finally, three 
points on tfie edge of the pin 46 are determined by 
tactile search and the exact pin center is located. 
The other two pin centers and normals are deter- 
mined in a similar manner, except that previously 

15 determined pin center and normal information may 
be combined with measured pin positions and nor- 
mals from the CT images to reduce the amount of 
manual guiding needed to teach approximate pin 
normals. The surgeon verifies the accuracy of the 

20 robot tool tip positioning before proceeding. This 
pin 46-related positional information is employed to 
compute the transformation between the CT-de- 
rived bone coordinates and the robot cutter 22 
coordinates. 

25 Next, the cutter 22 bit is placed into the collet 
and a further calibration check is made to verify 
that it is installed correctly. 

The robot 12 then mills out the correct shape 
to receive the implant. The surgeon monitors the 

30 progress of the operation both by direct visual 
observation and through observation of the online 
display 48 where CT derived images and selected 
cross-sections of the implant model are superim- 
posed. 

35 After the shape is cut the robot is withdrawn to 
a standby position and manipulator power is turned 
off. The femoral implant 1 is installed and the 
femoral fixator 44 and the aligning pins 46 are 
removed. The operation then proceeds to comple- 

40 tion in a conventional manner. 

It should be realized that the functionality de- 
scribed above could be further subdivided amongst 
additional interconnected processors or concentrat- 
ed into fewer processors. Also, although the inven- 

45 tion has been described in the context of a hip 
implant application the teaching of the invention is 
also applicable to other surgical procedures that 
either do or do not require the milling of a precise 
shape into bone tissue. Furthermore, alternative 

so embodiments incorporate additional measures both 
to reduce the robot's top speed and acceleration 
and to further increase the redundancy of checking. 
The former may be accomplished either by me- 
chanical modifications to the manipulator arm 14 or 

55 by imposing electrical limits on motor voltage or 
current. The latter measure includes the incorpora- 
tion of redundant position and velocity encoders 
monitored by a separate computer, preferably elec- 



8 



13 



EP0 456 103 A2 



14 



trically isolated from the robot controller 24. If an 
out-of-tolerance or out-of-control condition is de- 
tected the separate computer takes appropriate 
action, such as asserting a manipulator "freeze" 
signal or by turning off manipulator power. 

Claims 

1. A robotic surgical system comprising: 

multiple degree of freedom manipulator arm 
means including a surgical tool, the manipula- 
tor arm means being coupled to means for 
controllably positioning the surgical tool within 
a three dimensional coordinate system; and 

means for determining a position of the sur- 
gical tool in the three dimensional coordinate 
system relative to a volumetric model, the de- 
termining means including means for detecting 
a position of the surgical tool within the three 
dimensional coordinate system, the detecting 
means having an output coupled to means for 
processing the volumetric model for determin- 
ing the position of the surgical tool relative to 
the volumetric model. 

2. A system as set forth in Claim 1 wherein the 
volumetric model is located within a surgical 
patient and wherein the system further com- 
prises means for detecting a motion of the 
patient relative to the three dimensional coordi- 
nate system. 

3. A system as set forth in Claim 2 wherein the 
detecting means is comprised of strain gage 
means coupled between the surgical patient 
and a reference point within the three dimen- 
sional coordinate system. 

4. A system as set forth in Claim 3 wherein the 
reference point is located upon means for im- 
mobilizing a portion of the patient that includes 
the volume of space. 

5. A system as set forth in Claim 1 and further 
comprising means for measuring a magnitude 
of a force applied to the surgical tool. 

6. A system as set forth in Claim 5 wherein the 
measuring means includes means for resolving 
at a tip of the surgical tool an applied force 
and torque. 

7. A system as set forth in Claim 1 and further 
comprising means for determining a coordinate 
transformation for locating the volumetric 
model within the three dimensional coordinate 



system. 

8. A system as set forth in Claim 1 wherein the 
volumetric model corresponds to a prosthetic 

5 implant, the volumetric model additionally cor- 

responding to a surgical tool approach volume 
relative to the prosthetic implant. 

9. A system as set forth in Claim 1 wherein the 
w surgical tool is a cylindrical rotatable cutter. 

10. A system as set forth in Claim 1 wherein the 
volumetric model is represented as a construc- 
tive solid geometry (CSG) tree data structure. 

75 

11. A system as set forth in Claim 1 wherein the 
detecting means is comprised of imaging 
means disposed for imaging a region of space 
that includes at least a portion of the manipula- 

20 tor arm means. 

12. A system as set forth in Claim 11 wherein the 
volumetric model is represented as a construc- 
tive solid geometry (CSG) tree data structure 

25 and wherein the imaging means has an output 
coupled to means for processing the CSG tree 
data structure for determining if the surgical 
tool is positioned within the volumetric model. 

30 13. A system as set forth in Claim 5 and further 
comprising means, coupled to the force mea- 
suring means, for determining through a force 
compliance technique a location of at least one 
reference point within the three dimensional 

35 coordinate system. 

14, A system as set forth in Claim 1 and further 
comprising means for displaying the position 
of the surgical tool relative to the volumetric 
40 model, the displaying means including means 
for superimposing computed tomagraphy (CT) 
information with corresponding cross-sectional 
images of the volumetric model. 

45 15. A system as set forth in Claim 5 and further 
comprising means, coupled to the force mea- 
suring means, for halting a further motion of 
the surgical tool if a force exceeding a first 
threshold is measured and for de-energizing 

50 the manipulator arm means if a force exceed- 
ing a second, higher threshold is measured. 

16. Apparatus for preparing a cavity within a bone 
to receive a prosthetic implant, comprising: 

55 

multiple degree of freedom manipulator arm 
means including a rotating surgical tool, the 
manipulator arm means being coupled to 
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means for controllably positioning the surgical 
tool within a three dimensional robot coordi- 
nate system; 

means for immobilizing the bone within the 
three dimensional robot coordinate system; 

means for determining the position of the sur- 
gical tool in the three dimensional robot coordi- 
nate system relative to a volumetric model 
representative of a cavity to be formed within 
the bone, the determining means including im- 
aging means disposed for imaging a region of 
space that includes at least a portion of the 
manipulator arm means for determining if the 
surgical tool is positioned outside of the volu- 
metric model; and 

means coupled between the bone and the im- 
mobilizing means and having an output for 
indicating an amount of movement, if any, be- 
tween the bone and the immobilizing means, 
the indicating means measuring the movement 
in three dimensions. 

17. Apparatus as set forth in Claim 16 wherein the 
volumetric model is represented as a construc- 
tive solid geometry (CSG) tree data structure, 
and wherein the determining means includes 
means for processing the CSG tree data struc- 
ture for determining if the surgical tool is posi- 
tioned outside of the volumetric model and for 
suspending a further movement of the surgical 
tool if the surgical tool is determined to be 
positioned outside of the volumetric model. 

18. Apparatus as set forth in Claim 16 and further 
comprising means for measuring a magnitude 
of a force applied to the surgical tool and 
means for suspending a further movement of 
the surgical tool if the force is found to exceed 
a predetermined magnitude. 

19. Apparatus as set forth in Claim 16 wherein the 
movement indicating means is comprised of 
strain gage means, the apparatus further com- 
prising means coupled to the output of the 
strain gage means for suspending a further 
movement of the surgical tool if the strain gage 
means detects a movement that exceeds a 
predetermined amount. 

20. Apparatus as set forth in Claim 16 wherein the 
bone has a plurality of reference pins affixed 
thereto for defining points of a bone coordinate 
system, wherein a surgeon manually guides 
the surgical tool to at least a first one of the 
reference pins, and wherein the apparatus fur- 



ther comprises means for determining a trans- 
formation between the bone coordinate system 
and the three dimensional robot coordinate 
system, the transformation determining means 
5 having an input coupled to an output of the 

imaging means. 

21. Apparatus as set forth in Claim 20 wherein the 
transformation determining means further de- 
10 termines a transformation between the three 
dimensional robot coordinate system and a 
three dimensional coordinate system associ- 
ated with the imaging means. 

75 22. A method of surgically removing a quantity of 
tissue from a patient, the tissue being removed 
from a region having a predetermined three 
dimensional shape, the method comprising the 
steps of: 

20 

representing the three dimensional shape as a 
volumetric model or as a portion of a volu- 
metric model; 

25 determining a location of the volumetric model 

relative to the tissue that is to be removed; and 

operating a multiple degree of freedom ma- 
nipulator arm means having a surgical tool 
30 coupled thereto such that the tissue is re- 
moved, the step of operating including the 
steps of: 

detecting a position of the manipulator arm 
35 means; 

determining a position of the surgical tool rela- 
tive to the position of the manipulator arm 
means in accordance with a predetermined 
40 transformation function; 

processing the volumetric model for determin- 
ing if the determined position of the surgical 
tool is outside of the volumetric model; and 

45 

suspending a further movement of the surgical 
tool if the surgical tool is determined to be 
positioned outside of the volumetric model. 

50 23. A method as set forth in Claim 22 wherein the 
step of operating includes an additional step of 
monitoring an output of a strain gage means to 
determine if a movement of the patient has 
occurred and suspending a further movement 

55 of the surgical tool if a movement is detected. 

24. A method as set forth in Claim 22 wherein the 
step of operating includes an additional step of 
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monitoring an output of a force measuring 
means coupled to the surgical tool and sus- 
pending a further movement of the surgical 
tool if a force is measured that exceeds a first 
predetermined threshold. 

25. A method as set forth in Claim 24 wherein the 
step of monitoring includes an additional step 
of de-energizing the manipulator arm means if 
a force is measured that exceeds a second, 
higher predetermined threshold. 

26. A method as set forth in Claim 22 wherein the 
step of operating includes an additional step of 
periodically monitoring an operational status of 
the manipulator arm means and suspending a 
further movement of the surgical tool if the 
operational status is determined to be less 
than optimum. 

27. A method as set forth in Claim 22 wherein the 
tissue includes bone tissue, wherein the volu- 
metric model has a shape representative of a 
shape of a prosthetic implant, and wherein the 
step of representing the three dimensional 
shape as a volumetric model includes a step 
of: 

selecting a shape of a prosthetic implant from 
a library of prosthetic implant shapes; 

and wherein the step of determining a location 
of the volumetric model relative to the tissue 
that is to be removed includes a step of: 

interactively determining a position of the se- 
lected prosthetic implant shape relative to im- 
ages of the bone tissue. 

2a A method as set forth in Claim 27 and includ- 
ing an additional step of storing data for use 
during the step of processing, the stored data 
being expressive of the dimensions and shape 
of the selected prosthetic implant and data 
expressive of the determined position. 

29. A method as set forth in Claim 27 wherein the 
step of operating further includes a step of 
displaying in succession cross-sectional im- 
ages of the bone tissue each having a cor- 
responding cross-sectional view of the select- 
ed prosthetic implant superimposed thereon, a 
specific one of the cross-sectional bone tissue 

. images being selected for display as a function 
of the position of the surgical tool relative to 
the volumetric model. 

30. A method as set forth in Claim 22 wherein the 



tissue includes bone tissue, wherein the volu- 
metric model has a shape representative of a 
shape of a prosthetic implant, and wherein the 
step of representing the three dimensional 
5 shape as a volumetric model includes a step 

of. 

defining a shape of a prosthetic implant as a 
function at least of the shape of the bone 
ro tissue; 

and wherein the step of determining a location 
of the volumetric model relative to the tissue 
that is to be removed includes a step of: 

15 

interactively determining a position of the de- 
fined prosthetic implant shape relative to im- 
ages of the bone tissue. 

20 31. A surgical planning system, comprising: 

means for providing cross-sectional images of 
a region of tissue into which a structure is to 
be implanted; 

means for specifying a three dimensional re- 
presentation of the structure; and 

means, responsive to input from an operator of 
30 the system, for superimposing cross-sectional 
images of the representation of the structure 
upon selected cross-sectional images of the 
region of tissue at user-selectable locations, 
the superimposing means including means for 
35 displaying the superimposed images to the 

user. 

32. A system as set forth in Claim 31 wherein the 
specifying means includes means for generat- 
40 ing a volumetric model of the structure and 

means for providing coordinates for locating 
the volumetric model within a tissue-based co- 
ordinate system. 

45 33. A system as set forth in Claim 32 wherein the 
specifying means further includes means for 
identifying reference points within the images 
of the region of tissue for deriving the tissue- 
based coordinate system. 

50 

34. A system as set forth in Claim 32 and further 
comprising means for providing data expres- 
sive of at least the volumetric model and the 
tissue-based coordinates to a robotic surgical 
55 system, the robotic surgical system compris- 

ing: 

multiple degree of freedom manipulator arm 
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means including a surgical tool, the manipula- 
tor arm means being coupled to means for 
controllably positioning the surgical tool within 
a three dimensional surgical coordinate sys- 
tem; and 5 

means for determining a position of the sur- 
gical tool in the three dimensional surgical 
coordinate system relative to the volumetric 
model, the determining means including w 
means for transforming the tissue-based co- 
ordinate system into the surgical coordinate 
system for locating the volumetric model there- 
in. 

75 
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